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1.  Introduction 


The  meteorological  (MET)  error  budget  for  artillery  is  part  of  the  complete  artillery 
error  budget.  The  MET  error  budget  tables  currently  list  MET  errors  that  are  based 
on  data  from  radiosonde  observations  (RAOBs)  at  various  increments  of 
“staleness.”  Staleness  is  commonly  defined  in  terms  of  time  (e.g.,  “half-hour  or 
30-min  MET”)  or  equivalent  distance  from  the  RAOB  (or  weather  balloon)  launch 
site.  The  staleness  levels  denote  levels  of  error  extracted  from  extensive  sets  of  data 
generated  several  decades  ago.  Appendix  A  presents  3  sample  MET  error  budget 
sheets.  A  MET  error  budget  sheet  should  not  be  taken  as  meaning  any  individual 
RAOB  launched  at  the  specified  time  in  the  past  (e.g.,  1  h  ago)  will  have  the  exact 
errors  listed  in  that  respective  staleness  error  budget  sheet.  For  example,  a  balloon 
launched  an  hour  before  a  firing  or  other  event  may  have  greater  or  less  error  than 
the  1-h  listing  depending  on  weather  conditions,  terrain,  and  so  on.  Furthermore  the 
error  values  in  the  existing  tables  may  not  accurately  reflect  the  errors  likely  to 
occur  when  using  data  from  a  numerical  weather  prediction  (NWP)  model-based 
system 

The  US  Anny  Research,  Development,  and  Engineering  Center  (ARDEC),  Firing 
Tables  and  Ballistics  Division  asked  for  assistance  to  bring  their  artillery  MET  error 
budget  tables  up  to  date  to  account  for  expected  errors  when  NWP  model-based 
systems  such  as  the  Profiler  Virtual  Module  (PVM)  are  used  rather  than 
radiosondes.  The  current  artillery  error  budget  tables  are  based  on  the  expected 
errors  in  the  MET  parameters  as  a  function  of  time  or  equivalent  distance  from 
when  artillery  MET  messages  are  produced  using  RAOBs.  The  PVM  uses  a  version 
of  the  Weather  Research  and  Forecasting  (WRF)  model,  which  includes  pre-  and 
postprocessing  software.  Skamarock  et  al.  (2008)  provides  information  on  the 
basics  of  the  WRF  model  in  some  detail.  While  certain  parts  have  changed  and  new 
features  added  since  their  paper,  the  main  aspects  of  the  WRF  have  remained  much 
the  same.  More  recent  documentation  and  other  information  may  be  found  on  the 
WRF  website  (http://wrf-model.org/index.php).  Additionally,  representatives  of 
various  member  nations  within  NATO  have  expressed  a  need  for  model-based 
MET  error  budget  values  and  US  representatives  also  expressed  an  interest  in 
having  those  data  shareable  within  NATO.  A  freely  available  civilian  version  of  the 
WRF  model  to  generate  the  appropriate  values  will  allow  distribution  throughout 
NATO  and  other  partner  nations  without  potential  restrictions  that  could  arise  from 
extracting  data  from  an  operational  military  system. 

This  report  presents  an  analysis  and  results  of  the  differences  between  relevant 
atmospheric  variables  from  computer  MET  messages  (METCMs)  and  ballistic 
messages  for  surface  to  surface  trajectories  (METB3s)  produced  from  “soundings” 
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extracted  from  WRF  output  and  those  generated  from  coincident  RAOBs.  The 
differences  were  first  generated  for  each  case  or  pair  of  WRF  and  coincident  RAOB 
METCMs  and  METB3s  by  zone  for  all  zones  from  the  surface  to  the  highest  zone 
covered  by  the  RAOB  or  the  WRF  output,  whichever  was  lower.  The  mean,  mean 
absolute  error  (MAE),  standard  deviation,  and  root  mean  square  error  (RMSE)  of 
the  difference  in  the  atmospheric  variables  of  the  131  METCM  and  METB3  pairs 
were  computed  and  entered  into  tables  for  each  of  the  message  zones.  We  note  that 
“errors”  in  the  context  of  this  report  refer  to  differences  from  the  coincident  RAOBs 
despite  known  limitations  in  accuracy  of  RAOBs.  To  help  clarify  that  we  are  not 
comparing  these  data  with  the  exact  atmospheric  values,  from  here  on,  we  adopted 
the  terms  mean  difference  (MD),  mean  absolute  difference  (MAD),  standard 
deviation  of  the  differences  (SD),  and  root  mean  square  difference  (RMSD)  instead 
of  mean,  MAE,  standard  deviation,  and  RMSE,  respectively,  for  use  in  this  report. 

This  investigation  provided  model-based  MET  error  budget  values  using  an  open- 
source  version  of  WRF.  The  tables  showing  the  results  are  formatted  similarly  to 
traditional  RAOB-based  tables.  Consequently,  the  transition  to  the  model-based 
tables  should  not  require  any  significant  effort  on  the  part  of  the  user. 

2.  Procedure 


The  model-based  messages  for  this  study  were  generated  from  vertical  profiles  of 
MET  data  derived  from  WRF  output.  The  vertical  profiles  (or  soundings)  were 
extracted  from  WRF  Network  Common  Data  Fonn  output  files  via  a  NCAR 
Command  Language  script.  WRF,  version  3.7.1,  was  run  with  9/3/1 -km  horizontal 
grid  spacing  nested  domains.  The  comparisons  for  this  report  used  data  from  the  3- 
km  domain.  The  initial  and  boundary  conditions  for  the  WRF  model  were  derived 
from  Global  Forecast  System  (GFS)  0.5°  horizontal  grid  spacing  with  a  3-h  time 
interval.  Initialization  data  were  extracted  from  GFS  output  found  at  the  National 
Oceanic  and  Atmospheric  Administration,  National  Operational  Model  Archive 
and  Distribution  System  website  (http://nomads.ncdc.noaa.gov/data.php?name= 
access#hires  weather  datasets).  The  WRF  runs  for  the  analysis  of  this  report  did 
not  include  the  assimilation  of  observation  data  such  as  regional  RAOBs. 
Consequently,  the  results  should  provide  a  conservative  set  of  data  that  covers 
nearly  the  entirety  of  potential  model  errors  since  data  assimilation  will  likely 
improve  model  output  rather  than  degrade  it.  They  are  also  applicable  to  battlefield 
systems  that  may  not  always  have  access  to  observations.  Cogan  (2016)  contains 
information  on  the  configuration  of  WRF  used  here,  except  that  there  was  no  data 
assimilation  and  we  used  the  12-h  forecast. 
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The  RAOB-based  messages  were  produced  from  soundings  obtained  from  the 
University  of  Wyoming  weather  website  (http://www.weather.uwyo.edu/ 
upperair/sounding.html).  That  site  contains  WMO  soundings  in  several  formats 
including  text  as  used  here.  As  an  alternative,  if  a  particular  sounding  is  not  available 
on  the  primary  site,  a  National  Oceanic  and  Atmospheric  Administration  website 
(http://www.esrl.noaa.gov/raobs)  archives  WMO  data  in  text  format. 

2.1  Generation  of  MET  Messages 

Cogan  (2015)  describes  the  methods  employed  to  process  the  WRF-  and  RAOB- 
based  profiles  into  METCM  or  METB3  messages.  Information  on  the  METCM  and 
METB3  formats  and  structures  may  be  found  in  FM  3-09.15  (2007),  Blaha  and 
Potuzak  (2013)  as  well  as  in  STANAG  4082  (2000)  and  STANAG  4061  (2000)  on 
the  METCM  and  METB3,  respectively.  Overall,  131  cases  were  included  for  the 
MET  budget  computations,  where  each  case  consists  of  a  coincident  METCM  or  a 
METB3  derived  from  WRF  output  and  from  the  respective  RAOB.  Since  separate 
tables  were  computed  for  the  METCM  and  the  METB3,  each  type  had  131  cases. 
The  differences  for  each  zone  for  each  case  comprised  one  sample.  Statistical 
quantities  were  computed  for  all  zones  from  the  surface  though  the  topmost  zone 
reached  by  the  RAOB  or  WRF,  whichever  was  lower.  The  WRF  output  extends 
vertically  from  the  surface  or  very  close  to  it  (e.g.,  0  m  for  pressure,  2  m  for 
temperature,  and  10  m  for  wind)  to  a  little  above  20  km  above  mean  sea  level 
(MSL).  Consequently,  for  locations  near  sea  level,  the  WRF-based  profiles  will 
extend  up  through  METCM  zone  26  (19-20  km)  and  through  all  METB3  zones 
(through  zone  15  or  16-18  km).  Since  METCM  and  METB3  zones  are  in  heights 
above  ground  level  (AGL),  the  maximum  zone  may  be  less  than  26  (METCM)  or 
15  (METB3)  for  sites  with  higher  elevations.  For  example,  the  Amarillo,  Texas, 
site  has  an  elevation  of  1099  m  and  as  a  result  the  maximum  METCM  zone  was  25 
(18-19  km).  Since  the  METB3  extends  only  up  to  18  km,  a  complete  message  was 
produced.  In  many  cases,  the  RAOB  covered  all  or  nearly  all  32  zones  of  the 
METCM  from  the  surface  though  zone  3 1 .  However,  comparisons  were  only  made 
through  zone  26,  the  maximum  zone  from  the  WRF-based  profiles.  At  times,  the 
RAOB  ended  at  heights  less  than  20  km  and  even  below  18  km  resulting  in 
METCMs  or  METB3s  with  fewer  than  27  or  16  zones,  respectively.  The  lower 
zones  contained  all  the  cases,  that  is,  131  samples  each.  However,  since 
comparisons  between  the  WRF  output  and  RAOB  values  ended  at  the  highest 
RAOB-  or  WRF-based  zone  (whichever  was  lower),  there  are  fewer  samples  at 
higher  zones.  The  number  of  samples  is  given  later  in  the  report  (in  Tables  1  and  2 
in  Section  3  for  the  METCM  and  METB3,  respectively). 
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The  sites  selected  were  in  both  the  northern  and  southern  hemispheres,  and  included 
tropical,  middle,  and  high  latitude  locations.  They  cover  diverse  climatic,  seasonal, 
and  geographical  conditions.  Appendix  B  contains  a  list  of  the  52  sites  used  for  this 
analysis.  Some  sites  such  as  the  ones  in  the  United  States  and  Australia  were  used 
more  than  once,  while  others  such  as  those  from  South  America  were  only  used 
once. 

The  variables  selected  were  based  on  needs  expressed  by  ARDEC,  which  focus  on 
the  METB3  and  METCM  variables  of  pressure,  temperature,  wind  speed,  wind 
direction,  and  the  derived  variable  of  density.  In  addition,  vector  wind  speed 
difference  was  included  to  provide  an  indication  of  the  combined  effect  of  wind 
direction  and  speed  differences.  Pressure  does  not  appear  in  the  current  MET  error 
budget  tables  that  have  the  METB3  fonnat,  but  is  in  the  METCM  and  is  an 
important  component  of  density  and  consequently  was  part  of  this  study.  The  winds 
were  analyzed  by  computing  the  difference  in  wind  speed  and  calculating  the  vector 
difference  between  the  2  horizontal  wind  vectors  derived  from  wind  speed  and 
direction  in  the  2  METCMs.  This  latter  quantity  is  referred  to  as  the  vector  wind 
speed  (VW)  difference  in  this  report.  VW  difference  normally  is  a  better  indicator 
of  wind  differences  with  respect  to  the  effect  on  ballistic  trajectories,  since  even  a 
fairly  small  wind  direction  difference  could  have  a  noticeable  impact  if  the  wind 
speed  is  high.  Also,  note  that  all  VW  differences  are  necessarily  positive  and  hence 
MD  and  MAD  will  be  identical.  Also,  since  METCMs  have  virtual  temperature, 
those  values  were  used  instead  of  sensible  temperature.  For  density  calculations, 
virtual  temperature  provides  more  accurate  values,  especially  in  regions  of  higher 
humidity  often  found  at  or  near  the  surface.  Nevertheless,  at  higher  altitudes  or  drier 
geographical  regions,  the  difference  between  sensible  and  virtual  temperature  most 
often  is  very  small.  Differences  of  the  variables  were  computed  between  the  WRF 
and  RAOB  METCMs  and  METB3s  for  each  zone  for  each  case  (i.e.,  1  sample). 
For  all  cases,  the  computed  statistical  variables  were  MD,  MAD,  SD,  and  RMSD, 
plus  the  sample  size  for  each  zone. 

2.2  Generation  of  Statistics  from  MET  Messages 

Processing  the  various  model  and  sounding  messages  and  detennining  the  statistics 
from  these  data  involve  several  steps.  The  initial  step  is  to  load  the  data  into  a 
Microsoft  Access  database.  The  second  step  is  compute  the  bias  errors  for  the 
appropriate  model  and  sounding  comparisons.  The  final  step  is  to  use  Microsoft 
Excel  to  pivot  the  raw  bias  data  into  the  desired  error  (i.e.,  difference)  statistics. 
Appendix  C  has  the  explicit  formulas  used. 
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The  first  step  in  the  process  is  loading  the  data  into  Microsoft  Access.  A  custom 
Visual  Basic  for  Applications  (VBA)  script  was  created  that  would  read  the  model- 
based  output  and  corresponding  sounding  data  for  each  message  type  (METCM  or 
METB3),  output  type  (model  or  sounding),  and  date/time  indicator.  Based  on 
message  and  output  type,  individual  tables  were  created  in  Access  that 
corresponded  to  all  of  the  data  for  a  given  message/output  type  and  included  the 
data  represented  in  natural  units  (scaling  removed)  along  with  station  ID,  date,  time, 
and  line  or  zone  information.  This  processing  step  created  the  raw  data  for  the  bias 
calculations. 

The  second  step  employed  Access  to  compute  bias  data  from  the  raw  data  via  the 
use  of  the  Standard  Query  Language  (SQL)  inherent  in  databases.  Noting  that  the 
bias  data  are  the  differences  between  model  and  sounding  at  each  line  or  zone  value, 
we  link  the  model  and  sounding  data  for  a  given  message  type  (for  example, 
METB3)  together  using  a  SQL  INNER  JOIN,  where  the  JOIN  links  the  model  and 
sounding  tables  by  location,  date,  time,  and  line.  This  step  ensures  that  the  bias  data 
are  correctly  computed  for  each  message  type.  This  creates  a  table  organized  by 
location,  date,  time,  and  line  for  a  given  message  type  that  contains  the  bias  data 
for  each  point  compared.  A  SQL  SELECT  query  transforms  this  table  into  the  data 
table  from  which  we  compute  the  statistics.  This  step  also  adds  in  some 
supplementary  calculations  to  facilitate  pivot  operations  in  Excel,  which  are  used 
in  the  next  step  to  compute  the  range  of  statistics  desired  and  include  the  absolute 
value  and  square  of  the  bias  data. 

The  final  step  employs  Microsoft  Excel  to  create  pivot  tables  that  encapsulate  the 
statistics  organized  in  tabular  form  as  presented  in  the  next  section.  In  this  way,  we 
can  pivot  on  the  raw  bias  data  and  compute  the  average  by  line  to  arrive  at  the  mean 
deviation  or  MD  per  line.  Using  the  raw  and  supplementary  data  developed  in 
step  2,  we  can  create  the  range  of  statistical  measures  described  in  Appendix  C.  In 
practice,  where  the  original  data  were  given  in  coded  units,  we  add  in  a  small  fourth 
step  using  Excel  to  rescale  the  bias  statistics  back  into  the  original  fonn,  that  is,  if 
the  data  were  originally  coded  as,  for  example,  temperature  *  10  and  MD  was 
computed  for  temperature,  the  data  presented  in  the  following  section  would  be 
represented  as  MD  *  10. 

One  final  note  regarding  wind  difference  computations.  These  differences  are 
computed  in  tenns  of  the  easterly  (U)  and  northerly  (V)  flows.  This  allows  us  to 
compute  the  VW  bias  as 


VWbias  =  yj (JJWRF  -  URAOBy  +  (yWRF  _  yRAOBy ? 


(1) 
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in  addition  to  independent  calculations  on  wind  speed  and  direction. 

3.  Results 


Table  1  contains  statistical  summaries  for  the  METCM  variables  in  each  individual 
zone,  for  all  zones,  and  for  all  cases,  and  includes  the  sample  size.  The  decrease  in 
number  of  samples  from  the  131  cases  for  higher  zones  is  due  to  the  data  in  some 
RAOBs  or  WRF  profiles  ending  or  becoming  incomplete  before  reaching  the  upper 
boundary  height  of  these  higher  zones. 

An  initial  look  at  a  subset  of  the  data  showed  high  values  of  pressure  differences. 
These  large  differences  mostly  arose  from  the  differences  in  elevation  between  the 
actual  height  (MSL)  at  the  RAOB  site  and  the  height  derived  from  the  WRF 
model’s  terrain  database.  To  at  least  partially  account  for  the  elevation  differences, 
we  adjusted  the  surface  elevation  of  the  sounding  (WRF  or  RAOB)  with  the  lower 
value  to  that  of  the  higher  one,  but  did  not  change  the  sounding  itself.  Then  the 
software  interpolated  the  surface  values  from  the  sounding  with  the  lower  surface 
height  (MSL)  so  that  it  started  at  the  revised  (“new”)  elevation.  For  example,  if  the 
WRF  sounding  had  an  elevation  of  5  m  and  the  RAOB  had  one  of  40  m,  the  WRF 
sounding  would  be  modified  so  that  the  output  for  the  METCM  or  METB3  would 
begin  at  40  m  MSL.  Interpolation  is  preferred  over  downward  extrapolation  from 
the  surface  values  of  the  sounding  with  the  higher  surface  elevation  since  such 
extrapolation  requires  assumptions  such  as  a  standard  temperature  lapse  rate  or  no 
change  in  wind  direction.  Thus  the  accuracy  of  an  interpolated  sounding  most  likely 
will  be  better  compared  to  one  that  is  extrapolated. 

Table  2  contains  statistical  summaries  for  the  METB3  variables  in  each  individual 
zone  for  all  zones  for  all  cases  and  includes  each  zone's  sample  size.  The  statistics 
in  the  current  RAOB-based  tables  (Appendix  A)  are  different  from  those  in  WRF- 
based  (or  based  on  another  model)  tables.  The  total  error  (or  difference)  of  the 
current  RAOB-based  tables  is  the  square  root  of  the  sum  of  the  squares  of  the 
individual  errors  (i.e.,  instrument  error,  time  staleness  error,  and  distance  staleness 
error),  where  each  of  those  errors  are  assumed  to  be  independent  of  the  others. 
While  instrument  errors  are  not  directly  applicable  to  model-based  profiles,  if 
observations  are  assimilated  into  a  model  then  the  instrument  errors  can  contribute 
to  the  model  error  (this  WRF-based  study  did  not  include  the  assimilation  of 
observations).  The  distance  staleness  error  for  the  RAOBs  refers  to  the  distance 
between  the  balloon  launch  site  and  the  location  of  application  (e.g.,  the  horizontal 
midpoint  of  the  trajectory).  Model-based  profiles  have  a  related  source  of  error  in 
that  while  the  trajectory  is  impacted  by  the  weather  along  a  curved  line,  the  model 
provides  predictions  of  the  weather  in  boxes  as  defined  by  the  model’s  3-D  grid; 
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variations  within  any  given  box  are  not  represented.  Time  errors  can  apply  to  both 
RAOB  and  model-based  profiles;  the  difference  between  the  time  of  the  trajectory 
and  that  of  the  RAOB  or  the  time  represented  by  the  model  output  can  result  in 
differences  between  the  estimate  of  the  weather  at  the  trajectory  and  the  actual 
weather  at  the  trajectory.  A  major  source  of  error  in  a  model  comes  from  the 
accuracy  of  its  initial  conditions.  During  the  first  part  of  the  model  simulation,  error 
may  decrease  as  realistic  structures  are  “spun-up”  by  the  model  that  were  not 
represented  in  the  initial  conditions.  Additionally,  data  assimilation  may  be  applied 
and  result  in  lower  error  than  would  be  present  without  data  assimilation.  However, 
in  general,  the  error  grows  with  time.  This  growth  depends  on  aspects  such  as  the 
accuracy  of  the  numerical  schemes,  the  fidelity  of  the  representations  of 
atmospheric  physical  processes,  and  the  accuracy  of  the  boundary  conditions. 
Model  error  also  depends  on  the  resolution  of  the  model;  higher  resolution  is  able 
to  better  resolve  underlying  terrain  as  well  as  atmospheric  processes.  Stauffer 
(2013)  provides  further  details  on  sources  of  uncertainty  in  NWP.  Although  there 
are  differences  between  what  is  represented  by  the  model-based  RMSD  values  of 
Table  2  and  the  RAOB-based  “total  error”  values  in  Appendix  A,  they  may  be 
compared  in  that  both  are  representative  of  the  total  error  or  difference  in  the 
weather  parameters  produced  by  a  NWP  model  and  a  RAOB,  respectively. 

The  WRF  and  RAOB  temperature  and  density  are  converted  to  percentage  of  the 
respective  standard  atmosphere  value  prior  to  the  calculation  of  the  statistics.  The 
standard  atmosphere  for  METB3  messages  may  be  found  in  STANAG  4061  (2000) 
and  FM  3-09.15  (2007),  and  is  taken  from  the  1976  International  Civil  Aviation 
Organization  atmosphere  using  geopotential  heights.  Nearly  all  the  RMSDs  are 
smaller  than  the  total  errors  in  the  2.0-h  table  and  many  are  smaller  than  the 
respective  values  in  the  0.5-h  table.  The  exception  is  the  wind  speed  for  line  2, 
which  is  somewhat  larger  than  the  respective  value  in  the  2.0-h  table. 
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Table  1  Statistical  quantities  for  the  METCM  by  row  for  all  131  cases  with  respect  to  the  differences  from  the  RAOB  values.  Comparisons  were  made 
up  to  the  maximum  METCM  zone  covered  by  WRF  and  RAOB.  The  variables  were  differences  in  pressure  (mb  or  hPa),  virtual  temperature  (K*10), 
wind  speed  (kn),  and  wind  direction  (10s  of  mil). 


Pressure  (hPa) 

Virtual  Temperature  (K*10) 

Wind  Speed  (kn) 

Wind  Direction  (tens  of  mils) 

Line 

Samples 

MD 

MAD 

SD 

RMSD 

MD 

MAD 

SD 

RMSD 

MD 

MAD 

SD 

RMSD 

MD 

MAD 

SD 

RMSD 

0 

131 

-0.19 

1.05 

1.47 

1.48 

-5.7 

19.1 

24.6 

25.1 

2.29 

3.22 

3.92 

4.52 

24.06 

83.39 

117.49 

119.49 

1 

131 

-0.27 

0.98 

1.40 

1.42 

-4.5 

14.5 

18.1 

18.5 

2.60 

3.92 

4.57 

5.24 

12.63 

55.53 

84.20 

84.83 

2 

131 

-0.19 

0.88 

1.32 

1.33 

-1.9 

11.3 

14.3 

14.4 

0.73 

4.03 

5.34 

5.37 

13.76 

42.30 

66.28 

67.44 

3 

131 

-0.10 

0.77 

1.07 

1.07 

-2.0 

8.1 

10.4 

10.5 

-0.10 

3.38 

4.92 

4.90 

-5.26 

44.71 

73.34 

73.24 

4 

131 

-0.14 

0.67 

0.99 

1.00 

-0.6 

7.6 

9.6 

9.6 

-0.01 

3.46 

4.98 

4.96 

-3.44 

35.59 

53.63 

53.54 

5 

131 

-0.15 

0.62 

0.94 

0.95 

0.5 

7.7 

9.8 

9.7 

-0.32 

3.82 

5.40 

5.39 

-8.68 

29.66 

47.94 

48.54 

6 

131 

-0.05 

0.60 

1.05 

1.04 

0.5 

8.0 

10.1 

10.1 

-0.56 

3.74 

5.24 

5.25 

-10.39 

27.67 

50.82 

51.68 

7 

131 

-0.09 

0.60 

0.97 

0.97 

-0.9 

6.9 

8.8 

8.8 

-0.71 

3.96 

5.35 

5.38 

-1.92 

29.42 

52.81 

52.64 

8 

131 

-0.15 

0.55 

0.91 

0.92 

-0.7 

6.2 

8.0 

8.0 

0.03 

4.23 

6.10 

6.08 

3.23 

29.40 

49.53 

49.45 

9 

131 

0.00 

0.47 

0.79 

0.79 

0.6 

6.0 

7.7 

7.7 

0.17 

4.49 

6.30 

6.28 

-7.57 

23.19 

33.77 

34.48 

10 

131 

-0.08 

0.59 

0.91 

0.91 

-0.2 

5.1 

6.7 

6.7 

-0.04 

4.13 

5.80 

5.78 

-4.86 

20.97 

34.14 

34.36 

11 

131 

-0.04 

0.51 

0.82 

0.81 

0.6 

5.5 

6.8 

6.8 

-0.48 

4.50 

6.16 

6.16 

-3.84 

20.65 

38.03 

38.08 

12 

131 

-0.12 

0.55 

0.84 

0.85 

0.3 

5.5 

7.4 

7.4 

-1.06 

4.68 

6.72 

6.78 

-1.07 

17.45 

37.25 

37.12 

13 

131 

-0.03 

0.47 

0.76 

0.76 

-0.9 

6.4 

9.9 

9.9 

-1.28 

4.18 

5.80 

5.91 

-1.73 

19.04 

38.83 

38.72 

14 

131 

-0.08 

0.53 

0.78 

0.79 

-1.9 

6.8 

11.5 

11.7 

-1.02 

4.41 

6.14 

6.20 

1.02 

13.44 

20.71 

20.66 

15 

131 

-0.03 

0.46 

0.82 

0.82 

0.2 

6.9 

12.3 

12.3 

-1.22 

4.46 

6.27 

6.36 

3.18 

14.94 

28.42 

28.49 

16 

131 

-0.02 

0.43 

0.78 

0.78 

2.7 

7.9 

12.2 

12.4 

-1.60 

4.89 

6.99 

7.15 

1.44 

11.76 

18.65 

18.64 

17 

130 

0.05 

0.39 

0.82 

0.82 

6.4 

10.3 

13.0 

14.4 

-1.72 

4.52 

6.09 

6.31 

3.63 

12.23 

19.86 

20.12 

18 

130 

0.09 

0.42 

0.81 

0.81 

5.2 

11.0 

14.7 

15.5 

-1.61 

4.58 

6.51 

6.68 

4.15 

10.48 

17.47 

17.90 

19 

128 

-0.01 

0.27 

0.72 

0.71 

-2.9 

10.8 

14.5 

14.7 

-0.52 

5.09 

8.10 

8.09 

0.33 

10.08 

13.71 

13.66 

20 

127 

0.02 

0.35 

0.79 

0.79 

-2.8 

9.3 

12.6 

12.9 

-0.34 

4.87 

8.86 

8.83 

-1.62 

9.23 

14.83 

14.86 

21 

125 

0.04 

0.25 

0.65 

0.65 

-1.1 

10.4 

15.2 

15.2 

0.55 

4.84 

9.93 

9.91 

-2.22 

11.53 

21.03 

21.06 

22 

120 

0.05 

0.23 

0.63 

0.63 

1.0 

10.4 

17.2 

17.2 

-0.30 

4.60 

9.02 

8.98 

3.18 

20.94 

41.38 

41.33 

23 

116 

-0.04 

0.22 

0.69 

0.69 

1.9 

10.8 

19.6 

19.6 

-0.72 

4.32 

7.75 

7.75 

-8.15 

21.59 

45.17 

45.70 

24 

112 

-0.06 

0.19 

0.67 

0.67 

0.2 

11.5 

19.0 

18.9 

0.18 

4.30 

7.64 

7.61 

5.38 

32.93 

66.54 

66.46 

25 

106 

-0.04 

0.15 

0.39 

0.39 

-3.6 

8.7 

10.6 

11.1 

1.14 

4.24 

6.91 

6.97 

3.05 

25.29 

45.89 

45.77 

26 

72 

-0.04 

0.13 

0.35 

0.35 

-3.6 

10.6 

13.0 

13.4 

0.97 

4.47 

7.19 

7.20 

-6.26 

27.21 

46.28 

46.39 
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Table  2  Statistical  quantities  for  the  METB3  by  row  for  all  131  cases  with  respect  to  the  differences  from  the  RAOB  values.  Comparisons  were  made 
up  to  the  maximum  METB3  zone  covered  by  WRF  and  RAOB.  The  variables  were  differences  in  density  (%  standard),  temperature  (%  standard),  and 
wind  speed  (kn). 


Density  (%  Standard) 

Temperature  (%  Standard) 

Wind  Speed  (kn) 

Line 

Samples 

MD 

MAD 

SD 

RMSD 

MD 

MAD 

SD 

RMSD 

MD 

MAD 

SD 

RMSD 

0 

131 

0.17 

0.67 

0.87 

0.88 

-0.20 

0.67 

0.86 

0.88 

2.29 

3.22 

3.92 

4.52 

1 

131 

0.13 

0.52 

0.65 

0.66 

-0.16 

0.50 

0.63 

0.65 

2.60 

3.92 

4.57 

5.24 

2 

131 

0.08 

0.43 

0.54 

0.55 

-0.08 

0.40 

0.51 

0.51 

1.10 

3.76 

4.94 

5.04 

3 

131 

0.06 

0.34 

0.42 

0.42 

-0.09 

0.30 

0.38 

0.39 

0.15 

3.24 

4.55 

4.54 

4 

131 

0.04 

0.26 

0.34 

0.34 

-0.05 

0.23 

0.30 

0.31 

0.16 

2.80 

4.20 

4.18 

5 

131 

0.03 

0.23 

0.30 

0.30 

-0.03 

0.20 

0.27 

0.27 

-0.07 

2.85 

4.48 

4.46 

6 

131 

0.01 

0.19 

0.25 

0.25 

-0.02 

0.19 

0.25 

0.25 

-0.18 

2.74 

4.25 

4.24 

7 

131 

0.00 

0.17 

0.22 

0.22 

0.00 

0.16 

0.21 

0.21 

0.02 

2.82 

4.43 

4.41 

8 

131 

0.00 

0.14 

0.19 

0.19 

0.00 

0.13 

0.18 

0.17 

-0.05 

2.60 

4.13 

4.12 

9 

131 

0.00 

0.13 

0.17 

0.17 

0.00 

0.14 

0.18 

0.18 

-0.38 

2.56 

3.97 

3.98 

10 

131 

0.01 

0.12 

0.18 

0.18 

0.00 

0.14 

0.18 

0.18 

-0.75 

2.55 

3.58 

3.64 

11 

131 

-0.01 

0.12 

0.18 

0.18 

0.00 

0.14 

0.18 

0.18 

-0.73 

2.50 

3.27 

3.34 

12 

130 

-0.04 

0.12 

0.16 

0.17 

0.00 

0.14 

0.18 

0.18 

-0.90 

2.28 

3.07 

3.19 

13 

127 

0.00 

0.10 

0.14 

0.14 

0.00 

0.14 

0.18 

0.18 

-0.54 

2.13 

3.00 

3.03 

14 

120 

-0.01 

0.09 

0.14 

0.14 

0.01 

0.14 

0.18 

0.18 

-0.46 

2.16 

3.20 

3.22 

15 

112 

-0.02 

0.10 

0.14 

0.14 

0.00 

0.13 

0.18 

0.18 

-0.42 

1.99 

3.13 

3.14 

For  wind  speed,  Table  1  (METCM)  shows  a  layer  of  maximum  wind  speed  RMSD 
that  includes  zones  19-22  (12-16  km  AGL).  Higher  wind  speeds  at  those  altitudes 
are  reasonable  since  those  heights  are  near  the  tropopause,  which  normally  is  not 
far  from  where  jet  streams  often  occur.  Overall,  the  METB3  results  are  encouraging 
in  that  they  suggest  model-derived  MET  error  budgets  are  generally  within  the 
range  of  values  found  in  the  RAOB  based  0.5-  and  2.0-h  tables,  and  many  values 
may  be  close  to  or  smaller  than  the  0.5-h  values.  However,  the  fairly  consistent 
drop  off  of  wind  speed  difference  in  the  METB3  table  appears  to  be  inconsistent 
with  the  aforementioned  higher  wind  speed  differences  in  the  METCM  table  around 
zones  19-22.  An  initial  investigation  suggests  that  it  may  be  related  to  the 
weighting  of  lower  zones  in  the  METB3,  but  a  definitive  answer  remains  to  be 
detennined. 

As  noted  earlier,  vector  wind  speed  often  can  provide  a  better  indicator  of  wind 
difference  than  regular  wind  speed  alone,  since  it  involves  direction  differences  as 
well.  Table  3  presents  the  vector  wind  speed  difference  RMSDs  for  the  METCM 
and  the  METB3  zones.  The  respective  values  may  be  compared  with  the  wind  speed 
values  in  Tables  1  and  2. 
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Table  3  Vector  wind  speed  difference  RMSDs  (kn)  for  the  METCM  and  METB3  zones  in 
Tables  1  and  2.  Line  number,  midpoint  height  (m),  and  sample  size  are  given  as  well. 


METCM  METB3 


Line 

Height 

Samples 

RMSD 

Line 

Height 

Samples 

RMSD 

0 

0 

131 

6.53 

0 

0 

131 

6.53 

1 

100 

131 

7.44 

1 

100 

131 

7.44 

2 

350 

131 

8.03 

2 

350 

131 

7.57 

3 

750 

131 

7.70 

3 

750 

131 

6.97 

4 

1250 

131 

7.24 

4 

1250 

131 

6.19 

5 

1750 

131 

7.12 

5 

1500 

131 

5.89 

6 

2250 

131 

6.91 

6 

2500 

131 

5.37 

7 

2750 

131 

7.70 

7 

3500 

131 

5.73 

8 

3250 

131 

8.52 

8 

4500 

131 

5.41 

9 

3750 

131 

8.46 

9 

5500 

131 

5.15 

10 

4250 

131 

8.05 

10 

7000 

131 

4.78 

11 

4750 

131 

8.38 

11 

9000 

131 

4.41 

12 

5500 

131 

8.40 

12 

11000 

130 

4.21 

13 

6500 

131 

7.70 

13 

13000 

127 

3.79 

14 

7500 

131 

8.37 

14 

15000 

120 

3.90 

15 

8500 

131 

8.48 

15 

17000 

112 

3.73 

16 

9500 

131 

9.30 

17 

10500 

130 

9.20 

18 

11500 

130 

9.65 

19 

12500 

128 

9.51 

20 

13500 

127 

10.36 

21 

14500 

125 

11.21 

22 

15500 

120 

11.73 

23 

16500 

116 

9.79 

24 

17500 

112 

9.28 

25 

18500 

106 

8.27 

26 

19500 

72 

8.62 

A  further  consideration  is  the  comparison  with  coincident  RAOBs.  RAOBs  provide 
the  best  readily  available  approximations  to  the  real  atmosphere.  However,  aside 
from  instrument  errors,  there  are  spatial  errors  that  arise  from  the  drift  of  the 
radiosonde  as  it  ascends  and  temporal  errors  that  result  from  the  time  taken  from 
launch  to  end  of  sounding  (e.g.,  Seidel  et  al.  [2011])  as,  for  example,  when  the 
balloon  bursts.  Thus,  a  RAOB  represents  a  view  of  the  atmosphere  along  a  path 
defined  by  movement  of  the  radiosonde  in  space  and  time.  On  the  other  hand,  the 
WRF -based  soundings  represent  vertical  profiles  constructed  from  a  column  of 
3-km  x  3-km  “boxes”  with  different  thicknesses  at  a  specific  time.  Consequently, 
one  cannot  expect  an  exact  match  between  WRF-and  RAOB-derived  METCMs, 
and  neither  will  exactly  match  the  real  atmosphere.  Though  most  often  a  RAOB 
yields  a  closer  approximation  to  the  real  atmosphere,  a  coincident  WRF -based 
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sounding  may  be  a  closer  fit  for  a  specific  time  and  place,  at  least  for  one  or  more 
variables  over  a  part  of  the  vertical  extent. 

4.  Conclusion 


This  analysis  for  the  estimation  of  the  MET  error  budgets  from  numerical  model 
output  using  open-source  data  produced  statistics  for  the  variables  of  interest  for 
131  cases  using  METCMs  or  METB3s  generated  from  the  WRF  model  output 
compared  with  those  derived  from  coincident  RAOBs.  The  MDs,  MADs,  SDs,  and 
RMSDs  of  the  difference  values  were  computed  for  each  METCM  or  METB3  zone 
that  had  data  from  the  RAOB-  and  WRF-derived  METCMs  or  METB3s, 
respectively,  for  up  to  131  samples.  All  cases  were  included  in  the  lower  zones  with 
a  drop  off  in  the  number  of  samples  at  the  higher  zones  due  to  some  RAOBs  that 
ended  or  had  incomplete  data  before  reaching  the  higher  zone  altitudes  or  WRF- 
derived  profiles  that  ended  before  20  km  AGL.  For  the  METB3,  the  values  from 
this  study  suggest  that  the  WRF-based  METB3s  mostly  have  values  that  are  smaller 
than  the  respective  ones  for  the  current  2.0-h  MET  error  budget  tables  and  many 
are  smaller  than  the  ones  from  the  0.5-h  table.  However,  more  exact  comparisons 
between  values  in  the  WRF-  and  RAOB-based  tables  are  made  more  difficult  due 
to  uncertainties  in  the  exact  methodology  used  to  create  the  RAOB-based  tables. 

Error  in  the  context  of  this  report  refers  to  the  difference  between  METCM  or 
METB3  values  computed  using  WRF  output  as  the  source  of  data  and  those 
computed  from  the  coincident  RAOBs.  Since  a  RAOB  is  not  actually  derived  from 
measurements  of  the  atmosphere  directly  above  a  specified  location  on  the  ground 
at  a  specified  time  due  to,  for  example,  balloon  drift  and  duration  of  ascent,  a 
complete  description  of  the  MET  error  budget  should  also  account  for  the  potential 
error  inherent  in  RAOB-derived  METCMs  or  METB3s.  The  model-based  error 
budgets  presented  here  are  based  on  a  specific  configuration  of  a  specific  model 
(WRF)  and  actual  error  budgets  may  vary  depending  on  the  model  employed  and 
the  exact  configuration  used.  Furthermore,  these  results  represent  an  average  over 
a  variety  of  cases;  the  errors  in  a  particular  situation  may  vary  substantially  from 
those  presented  herein  due  to  case-to-case  variations. 

Nevertheless,  this  analysis  provides  a  basis  for  further  work  and  a  way  forward  for 
a  more  complete  compilation  of  these  statistics  for  a  MET  error  budget  based  on 
numerical  weather  model  output. 
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Appendix  A.  Sample  Meteorological  (MET)  Error  Budget  Tables 
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This  Appendix  presents  3  sheets  listing  the  meteorological  (MET)  error  budget 
tables  for  3  staleness  levels:  0.5  h  and  10  km,  which  has  the  least  error;  2  h  and 
20  km;  and  4  h  and  48  km  (Tables  A-l  through  A-3).  The  tables  show  the  error  due 
to  instrument  error,  time  staleness  error,  spatial  staleness  error,  and  the  total  error 
from  these  3  contributions.  A  complete  set  of  MET  budget  data  sheets  were 
provided  by  the  US  Army  Armament  Research,  Development,  and  Engineering 
Center  in  an  unpublished  document. 1  Time  and  distance  staleness  are  listed  on  each 
table  in  the  upper-left  corner.  The  MET  lines  are  those  of  the  ballistic  MET  message 
for  surface-to-surface  fires  (METB3).  As  noted  in  the  upper-right  comer  of  each 
table,  the  MET  budget  used  the  mean  mid-latitude  wind  profile.  Presumably, 
radiosonde  observations  (RAOBs)  at  different  temporal  and  spatial  separations 
from  the  verification  RAOB  are  compared  to  the  verification  RAOB  to  calculate 
differences.  The  density  and  temperature  values  are  converted  to  respective 
percentages  of  the  1976  International  Civil  Aviation  Organization  standard 
atmosphere.  These  tables  were  computed  for  the  METB3  message  and  have 
METB3  units  for  the  3  variables  listed  (i.e.,  percentage  of  standard  for  density  and 
temperature,  and  knots  for  wind).  Note  that  the  values  for  line  0  (not  shown  on  these 
tables)  are  the  same  as  for  line  1 . 


*Met  error  in  the  delivery  accuracy  error  budget.  Picatinny  Arsenal  (NJ):  US  Army  Armament  Research, 
Development,  and  Engineering  Center.  Unpublished,  2013. 
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Table  A-l  MET  budget  values  for  the  listed  time  (0.5  h)  and  separation  (10  km)  staleness 
values 


Time 

Staleness 

(hr) 

[0.5  to  12] 

Separator! 

(Distance) 

Staleness 

(km) 

0.5 

10 

Mid  Latitude  Mean  Wind  Profile 
Optimized  for  Annual  Winds  (FARMY- 153-5) 
NAVAIDS  (MDS)  Used  for  Instrument  Error 


Density 

Wind 

Temperature 

Met 

Line 

Instrument 

Error 

_ 

Time 

Staleness 

Error 

!%1 

Distance 

(Separation 

and 

Balloon 

Drift) 

Staleness 

Error 

m 

Total 

Error 

BH 

Instrument 

Error 

<*»> 

Time 

Staleness 

Error 

03=) 

Distance 

(Separation 

and 

Balloon 

Drift) 

Staleness 

Error 

(kK) 

Total 

Error 

(■as) 

Instrument 

Error 

_ 

Time 

Staleness 

Error 

!*) 

Distance 

(Separation 

and 

Balloon 

Drift) 

Staleness 

Error 

_ 

Total 

Error 

ea 

1 

0.15 

0.65 

0.53 

0.85 

1.65 

1.84 

3.03 

3.91 

0.25 

0.50 

0.41 

0.69 

2 

0.15 

0.41 

0.34 

0.55 

1.04 

1.90 

1.55 

2.66 

0.25 

0.31 

0.26 

0.48 

3 

0.15 

0.29 

0.24 

0.40 

0.55 

1.99 

1.63 

2.63 

0.25 

022 

0.18 

0.38 

4 

0.15 

0.24 

0.20 

0.34 

0.42 

2.05 

1.69 

2.70 

0.25 

0.18 

0.15 

0.34 

5 

0.15 

0.21 

0.17 

0.31 

0.41 

2.10 

1.75 

2.76 

0.25 

0.16 

0.13 

0.32 

6 

0.15 

0.17 

0.14 

0.27 

0.25 

2.14 

1.84 

2.83 

0.25 

0.13 

0.11 

0.30 

7 

0.15 

0.15 

0.13 

0.25 

0.21 

2.16 

1.97 

2.93 

0.25 

0.11 

0.10 

0.29 

8 

0.15 

0.13 

0.13 

0.24 

0.17 

2.17 

2.13 

3.05 

0.25 

0.10 

0.10 

0.29 

9 

0.15 

0.12 

0.13 

0.23 

0.13 

2.21 

2.36 

3.23 

0.25 

0.09 

0.10 

0.28 

10 

0.15 

0.10 

0.13 

0.22 

0.11 

2.41 

2.96 

3.81 

0.25 

0.08 

0.10 

0.28 

11 

0.15 

0.09 

0.13 

0.22 

0.07 

2.47 

3.60 

4.36 

0.25 

0.07 

0.10 

0.28 

12 

0.15 

0.08 

0.14 

0.22 

0.06 

2.42 

4.08 

4.74 

0.25 

0.06 

0.11 

0.28 

13 

0.15 

0.08 

0.15 

0.22 

0.05 

2.20 

4.11 

4.66 

0.25 

0.06 

0.11 

0.28 

14 

0.15 

0.07 

0.15 

0.22 

0.04 

1.93 

3.87 

4.32 

0.25 

0.06 

0.11 

0.28 

15 

0.15 

0.07 

0.14 

0.22 

0.03 

1.65 

3.45 

3.82 

0.25 

0.05 

0.11 

0.28 

Table  A-2  MET  budget  values  for  the  listed  time  (2  h)  and  separation  (20  km)  staleness  values 


Time 

Staleness 

(hr) 

[0.5  to  12] 

Separaton 

(Distance) 

Stateness 

(km) 

2 

20 

Mid  Latitude  Mean  Wnd  Profile 
Optimized  for  Annual  Winds  (FARMY- 153-5) 
NAVAIDS  (MDS)  Used  for  instrument  Error 


Density 

Wine 

Temperature 

Met 

Line 

Instrument 

Error 

_ 

Time 

Staleness 

Error 

_ 

Distance 

(Separation 

anc 

Balloon 

Drift) 

Staleness 

Error 

m 

Total 

Error 

c»> 

Instrument 

Error 

(ks) 

Time 

Staleness 

Error 

(k»s) 

Distance 

(Separation 

and 

Ballcon 

Drift) 

Staleness 

Error 

_ 

Total 

Error 

(ktt) 

Instrument 

Error 

ey 

Time 

Staleness 

Error 

_ 

Distance 

(Separation 

and 

Balloon 

Drift) 

Staleness 

Error 

_ 

Tota 

Error 

w 

1 

0.10 

1.30 

0.73 

1.01 

1.00 

3.08 

3.47 

5.32 

a.  25 

0.99 

0.57 

1.18 

2 

0.15 

0.82 

0.48 

0.96 

1.34 

3.60 

2.19 

4.51 

a.  25 

0.83 

0.36 

0.77 

3 

0.15 

0.58 

0.34 

0.69 

0.55 

3.68 

2.30 

4.63 

a.  25 

0.44 

0.26 

0.57 

4 

0.15 

0.47 

0.27 

0.57 

0.42 

4.11 

2.38 

4.76 

0.25 

0.36 

021 

0.49 

5 

0.15 

0.41 

0.24 

0.50 

0.41 

4.19 

2.44 

4.87 

025 

031 

0.18 

0.44 

6 

0.15 

0.34 

0.20 

0.42 

025 

4.27 

2.50 

4.96 

025 

0.26 

0.15 

0.30 

7 

0.15 

0.29 

0.17 

0.37 

021 

4.31 

2.58 

5.03 

025 

0.22 

0.13 

0.36 

8 

0.15 

0.26 

0.16 

0.34 

0.17 

4.33 

2.66 

5.09 

025 

020 

0.12 

0.34 

9 

0.15 

0.24 

0.15 

0.32 

0.13 

4.41 

2.81 

5.23 

025 

0.18 

0.12 

0.33 

10 

0.15 

0.21 

0.14 

0.29 

0.11 

4.61 

3.32 

5.85 

025 

0.16 

0.11 

0.31 

11 

0.15 

0.18 

0.14 

0.28 

007 

4.63 

3  84 

6.25 

025 

0.14 

0.11 

0.31 

12 

0.15 

0.17 

0.15 

0.27 

006 

4.64 

4.24 

6.43 

025 

0.13 

0.11 

0.30 

13 

0.15 

0.16 

0.15 

0.26 

005 

4.39 

4.22 

6.09 

025 

0.12 

0.11 

0.30 

14 

0.15 

0.15 

0.15 

0.26 

004 

3.65 

3.94 

5.51 

025 

0.11 

0.11 

0.30 

15 

0.15 

0.14 

0.15 

0.25 

003 

3.30 

3.51 

4.82 

025 

0.10 

0.11 

0.20 
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Table  A-3  MET  Budget  values  for  the  listed  time  (4  h)  and  separation  (48  km)  staleness 
values 


Time 

Staleness 

(hr) 

[0.5  to  12] 

Separatton 

(Distance) 

Staleness 

(km) 

4 

48 

Mid  Latitude  Mean  Wind  Profile 
Optimized  for  Annual  Winds  (FARMY-153-5) 
NAVAIDS  (MDS)  Used  for  instrument  Error 


Density 

Wind 

Temperature 

Met 

Line 

Instrument 

Error 

(%) 

Time 

Staleness 

Error 

_ 

Distance 

(Separation 

and 

Balloon 

Drift) 

Staleness 

Error 

_ 

Total 

Error 

<») 

Instrument 

Error 

(kts) 

Time 

Staleness 

Error 

(tos) 

Distance 

(Separation 

and 

Balloon 

Drift) 

Staleness 

Error 

(i«»»> 

Total 

Error 

(to) 

Instrument 

Error 

(%) 

Time 

Stateness 

Error 

(%) 

Distance 

(Separation 

and 

Balloon 

Drift) 

Stateness 

Error 

(%) 

Total 

Error 

(%) 

1 

0.15 

1.84 

1.16 

2.18 

1.65 

5.20 

3.89 

6.70 

0.25 

1.41 

0.89 

1.68 

2 

0.15 

1.10 

0.74 

1.38 

1.04 

5.37 

3.40 

6.44 

0.25 

0.89 

0.56 

1.08 

3 

0.15 

0.82 

0.52 

0.98 

0.55 

5.63 

3.56 

6.69 

0.25 

0.63 

0.40 

0.79 

4 

0.15 

0.67 

0.43 

0.81 

0.42 

5.81 

3.67 

6.89 

0.25 

0.51 

0.33 

0.66 

5 

0.15 

0.58 

0.37 

0.70 

0.41 

5.93 

3.75 

7.03 

0.25 

0.44 

0.28 

0.58 

6 

0.15 

0.47 

0.30 

0.58 

0.25 

6.04 

3.83 

7.16 

0.25 

0.36 

0.23 

0.50 

7 

0.15 

0.41 

0.26 

0.51 

0.21 

6.10 

3.88 

7.24 

0.25 

0.31 

020 

0.45 

8 

0.15 

0.37 

0.24 

0.46 

0.17 

6.13 

3.92 

7.28 

0.25 

0.28 

0.18 

0.42 

9 

0.15 

0.34 

0.22 

0.43 

0.13 

6.24 

4.03 

7.43 

0.25 

0.26 

0.17 

0.39 

10 

0.15 

0.2© 

0.19 

0.38 

0.11 

6.81 

4.49 

8.15 

0.25 

0.22 

0.15 

0.37 

11 

0.15 

0.26 

0.18 

0.35 

0.07 

6.97 

4.80 

8.46 

0.25 

020 

0.14 

0.35 

12 

0.15 

0.24 

0.17 

0.33 

0.06 

6.84 

4.97 

8.46 

0.25 

0.18 

0.13 

0.34 

13 

0.15 

0.22 

0.17 

0.32 

0.05 

6.21 

4.76 

7.82 

0.25 

0.17 

0.13 

0.33 

14 

0.15 

0.21 

0.16 

0.30 

0.04 

5.45 

4.35 

6.97 

0.25 

0.16 

0.13 

0.32 

15 

0.15 

0.1© 

0.16 

0.29 

0.03 

4.66 

3.83 

6.03 

0.25 

0.15 

0.12 

0.32 
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Appendix  B.  World  Meteorological  Organization  (WMO)  Sites 
Used  for  the  Meteorological  (MET)  Error  Budget  Analysis 
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Table  B-l  presents  the  complete  list  of  the  World  Meteorological  Organization 
(WMO)  sites  used  for  the  meteorological  (MET)  error  budget  analysis.  A  total  of 
52  sites  were  selected  that  cover  a  variety  of  geographical  and  climatological 
regions  for  all  seasons,  though  the  largest  number  of  cases  were  from  November 
2015  through  June  2016. 

Table  B-l  List  of  all  WMO  radiosonde  observation  (RAOB)  stations  used  in  the  analysis  of 
this  report.  Elevations  are  in  meters  above  mean  sea  level  (MSL). 


Region 

Station 

Latitude 

Longitude 

Elevation 

(degrees) 

(degrees) 

(m) 

US  ■  Southwest 

FGZ  (Flagstaff,  AZ) 

35.23 

-111.82 

2192 

NKX  (San  Diego,  CA) 

32.85 

-117.12 

137 

TUS  (Tucson,  AZ) 

32.23 

-110.96 

751 

VEF  (Las  Vegas,  NV) 

36.05 

-115.18 

697 

Wfo-PHX  (Phoenix,  AZ) 

33.45 

-111.95 

384 

US  -  East  Coast 

GSO  (Greensboro,  NC) 

36.08 

-79.95 

270 

IAD  (Dulles  AP,  VA) 

38.98 

-77.46 

93 

MHX  (Newport,  NC) 

34.78 

-76.88 

11 

RNK  (Blacksburg,  VA) 

37.20 

-80.41 

654 

WAL  (Wallops  Is.,  VA) 

37.93 

-75.47 

12 

US-Mid-West 

AMA  (Amarillo,  TX) 

35.23 

-101.70 

1099 

DDC  (Dodge  City,  KS) 

37.76 

-99.97 

790 

LMN  (Lamont,  OK) 

36.62 

-97.48 

317 

OUN  (Norman,  OK) 

35.18 

-97.44 

345 

TOP  (Topeka,  KS) 

39.07 

-95.62 

270 

US-NE/Canada 

ALB  (Albany,  NY) 

42.69 

-73.83 

95 

GYX  (Gray,  ME) 

43.90 

-70.25 

125 

WMW  (Maniwaki,  Quebec) 

46.30 

-76.01 

189 

US-Mountain 

GJT  (Grand  Junction,  CO) 

39.11 

-108.53 

1475 

RIW  (Riverton,  WY) 

43.06 

-108.48 

1703 

SLC  (Salt  Lake  City,  UT) 

40.77 

-111.95 

1289 

Germany 

ETGB  (Bergan) 

52.81 

9.93 

69 

ETGI  (Idar-Oberstein) 

ETGK 

49.70 

7.33 

377 

(Kuemmersbruck) 

49.43 

11.90 

418 

MEIN  (Meiningen) 

50.56 

10.38 

450 

STUT  (Stuttgart) 

48.83 

9.20 

321 

South  Korea 

CHEJ  (Cheju) 

33.28 

126.16 

73 

HEUK  (Heuksando) 

34.68 

125.45 

69 

POHA(Pohang) 

36.03 

129.38 

6 

RKJJ  (Kwangju) 

35.11 

126.81 

13 

RKSO(OsanAB) 

37.10 

127.03 

52 

Australia 

YMML  (Melbourne) 

-37.66 

144.85 

119 

YPAD  (Adelaide) 

-34.95 

138.53 

4 

Alaska 

PADQ  (Kodiak) 

57.75 

-152.50 

34 

PAKN  (King  Salmon) 

58.68 

-156.67 

8 

PANC  (Anchorage) 

61.16 

-150.01 

40 

East  Europe 

BU DA  (Budapest,  Hungary) 

47.43 

19.18 

139 

POP  (Poprad-Gabivce,  Slovakia) 

49.03 

20.31 

706 

PRA  (Prada-Libus,  Czech  -Rep) 

50.01 

14.45 

304 

PRO  (Prostejov,  Czech  -Rep) 

49.45 

17.13 

216 
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Table  B-l  List  of  all  WMO  radiosonde  observation  (RAOB)  stations  used  in  the  analysis  of 
this  report.  Elevations  are  in  meters  above  mean  sea  level  (MSL)  (continued). 


NE  Brazil 

FORT  (Fortaleza) 

SB  NT  (Natal 

-3.76 

-38.60 

19 

Aer  oporto) 

-5.91 

-35.25 

49 

South  Brazil 

SB  PA  (Porto  Alegre) 

-30.00 

-51.18 

3 

SBSM  (Santa  Maria) 

-29.72 

-53.70 

85 

SE  Brazil 

SBFL  (Florianopolis) 

-27.67 

-48.55 

5 

SBLO  (Londrina) 

-23.33 

-51.13 

569 

SBMT  (Marte  Civ) 

-23.52 

-46.63 

722 

SCBT  (Curitiba  Aeroporto) 

-25.51 

-49.16 

908 

Argentina 

SAME  (Mendoza) 

-32.83 

-68.78 

704 

Chile 

SCSN  (Santo  Domingo) 

-33.65 

-71.61 

75 

South  Africa 

FAIR  (Pretoria) 

-25.91 

28.21 

1523 

TALC  (King  Shaka) 

-29.61 

31.12 

109 
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Appendix  C.  Error  Calculations 
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This  Appendix  describes  the  specific  calculations  for  computing  the  error  (i.e., 
difference)  statistics  used  throughout  this  report. 1 

The  following  are  the  definitions  used. 

X[  is  the  value  of  an  observed  meteorological  variable,  selected  from  a  sounding  or 
model  output,  expressed  in  natural  units  (e.g.,  K,  or  hPa). 

Yl  is  a  particular  instance  of  Xi  stored  as  a  coded  meteorological  variable  in  a 
sounding  or  model  output  file  (e.g.,  Yi  might  take  the  value  of  10  *  Xh  where  Xt  in 
this  case  represents  temperature). 

Let  X[  —  kj1  Yi  ,  where  kj  >  0  is  an  appropriately  chosen  scaling  factor  that  may 
depend  on  altitude  but  not  on  the  observation  itself.  For  the  computer 
meteorological  (MET)  message  (METCM)  the  factor  k  =  10  for  temperature  or 
virtual  temperature  where  the  units  are  tenths  of  a  degree  (e.g.,  2915  instead  of 
291.5)  and  0.1  for  wind  direction  with  units  of  tens  of  mils  (e.g.,  346  [rounded 
value]  instead  of  3458).  Similarly  for  the  ballistic  MET  message  for  surface-to- 
surface  fires  (METB3),  k  =  10  since  the  units  are  in  tenths  of  a  percent  (e.g.,  963 
instead  of  96.3). 

Assume  that  both  model  and  corresponding  sounding  data  are  scaled  by  the  same 
value  kj-,  thus,  the  bias  becomes 

Bi  =  X™  -  X?  =  kj1  (Ytm  -  Yf ),  (C-l) 

where  the  superscripts  m  and  s  indicate  a  model  or  sounding  value. 

As  noted  previously,  the  term  “deviation”  is  used  to  indicate  the  difference  or 
departure  of  measured  from  model  soundings. 

With  the  bias  now  defined  in  terms  of  either  natural  or  coded  MET  data,  we  can 
now  define  the  following: 

•  Mean  bias  =  mean  error  =  mean  deviation  — >  MD 

•  Mean  absolute  error  =  mean  absolute  deviation  -»  MAD 

•  Sample  standard  deviation  of  the  error  =  sample  standard  deviation  of  the 
bias  ->  SD 

•  Root  mean  square  error  =  root  mean  square  deviation  =  — >  RMSD 


*  Australian  Bureau  of  Meteorology.  WWRP/WGNE  joint  working  group  on  forecast  verification 
research.  Melbourne  (Australia):  Australian  Bureau  of  Meteorology;  2015  Jan  26  [accessed  2016], 
http://cawcr.gov.au/projects/verification/. 
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•  N  to  be  the  number  of  sample  points  under  consideration. 
The  error  statistics  are  computed  using  the  following  equations: 


MD  =  -jZ?  B,  =  -  If)  .  (C-2) 

MAD  =  -  r,sl  .  (C-3) 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


AGL 

above  ground  level 

ARDEC 

US  Army  Research,  Development,  and  Engineering  Center 

MAD 

mean  absolute  difference  or  mean  absolute  deviation 

MAE 

mean  absolute  error 

MD 

mean  difference  or  mean  deviation 

MET 

meteorological 

METB3 

ballistic  meteorological  message  for  surface-to-surface  fires 

METCM 

computer  MET  message 

MSL 

mean  sea  level  as  in  above  mean  sea  level 

NWP 

numerical  weather 

NATO 

North  Atlantic  Treaty  Organization 

PVM 

Profiler  Virtual  Module 

RAOB 

radiosonde  observation 

RMSD 

root  mean  square  difference  or  root  mean  square  deviation 

RMSE 

root  mean  square  error 

SD 

standard  deviation 

STANAG 

NATO  Standardization  Agreement 

WRF 

Weather  Research  and  Forecasting 

vw 

vector  wind  speed 
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